ESTIMATING THE COVARIANCE OF RANDOM MATRICES 
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Abstract. We extend to the matrix setting a recent result of Srivastava-Vershynin 
ETI about estimating the covariance matrix of a random vector. The resuh can be in- 
terpreted as a quantified version of the law of large numbers for positive semi-definite 
matrices which verify some regularity assumption. Beside giving examples, we dis- 
cuss the notion of log-concave matrices and give estimates on the smallest and largest 
eigenvalues of a sum of such matrices. 



1. Introduction 

In recent years, interest in matrix valued random variables gained momentum. Many 
of the results dealing with real random variables and random vectors were extended to 
cover random matrices. Concentration inequalities like Bernstein, Hoeffding and others 
were obtained in the non-commutative setting ( Il5l. [|22]| . [[T4]| ). The methods used were 
mostly combination of methods from the real/vector case and some matrix inequalities 
like the Golden-Thompson inequality (see f8l|). 

Estimating the covariance matrix of a random vector has gained a lot of interest re- 
cently. Given a random vector X in M", the question is to estimate S = EXX*. A 
natural way to do this is to take Xi, .., independent copies of X and try to approxi- 
mate S with the sample covariance matrix Sat = ;^ I]i XiXj. The challenging problem 
is to find the minimal number of samples needed to estimate S. It is known using a 
result of Rudelson (see [|T9l ) that for general distributions supported on the sphere of 
radius ^Jn, it suffices to take cn\og(n) samples. But for many distributions, a number 
proportional to n is sufficient. Using standard arguments, one can verify this for gauss- 
ian vectors. It was conjectured by Kannan- Lovasz- Simonovits ifTTll that the same result 
holds for log-concave distributions. This problem was solved by Adamczak et al ([I3]|, 
dUl). Recently, Srivatava-Vershynin proved in [|2TI covariance estimate with a number 
of samples proportional to n, for a larger class of distributions covering the log-concave 
case. The method used was different from previous work on this field and the main idea 
was to randomize the sparsification theorem of Batson-Spielman-Srivastava flTJ. 

Our aim in this paper is to adapt the work of Srivastava-Vershynin to the matrix setting 
replacing the vector X in the problem of the covariance matrix by an x m random 
matrix A and try to estimate EAA* by the same techniques. This will be possible since in 
the deterministic setting, the sparsification theorem of Batson-Spielman-Srivastava Q 
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has been extended to a matrix setting by De Carli Silva-Harvey-Sato [9] who precisely 
proved the following: 

Theorem 1.1. Let Bi, . . . , be positive semi-definite matrices of size n x n and 
arbitrary rank. Set B := J2i Bi. For any e G (0, 1), there is a deterministic algorithm 
to construct a vector y G M"^ with 0{n/e^) nonzero entries such that y > and 

B ^ Y.y^B^ ^ {l + e)B. 

i 

For an n X n matrix A, denote by ||y4|| the operator norm of A seen as an operator on 
• The main idea is to randomize the previous result using the techniques of Srivastava- 
Vershynin [l2T1l . Our problem can be formulated as follows: 



Take B a positive semi-definite random matrix of size nx n. How many independent 
copies of B are needed to approximate KB i.e taking Bi, Bjy independent copies of 
B, what is the minimal number of samples needed to make J2i Bi — KB very small. 

One can view this as an analogue to the covariance estimate of a random vector by 
taking for B the matrix AA^ where A is an n x m random matrix. With some regularity, 
we will be able to take proportional to n. However, in the general case this is no 
longer true. In fact, take B uniformly distributed on {neicj}-^^ where ej denotes the 
canonical basis of M". It is easy to verify that KB = J„ and Y,i Bi is a diagonal 
matrix and its diagonal coefficients are distributed as 

^(Pl,--,Pn), 

where pi denotes the number of times e^e* is chosen. This problem is well- studied and 
it is known (see lfT2]| ) that we must take N ^ cn log(n). This example is essentially due 
to Aubrun H. More generally, if 5 is a positive semi-definite matrix such that KB = In 
and Tr(i?) ^ n almost surely, then by Rudelson's inequality in the non-commutative 
setting (see IITSlI ) it is sufficient to take cn \og(n) samples. 

The method will work properly for a class of matrices satisfying a matrix strong 
regularity assumption which we denote by (M SR) and can be viewed as an analog to 
the property (SR) defined in [1211 . 

Definition 1.2. [Property (MSR)] 

Let B be an n X n positive semi-definite random matrix such that KB = In- We will say 
that B satisfies (MSR) if for some r] > Owe have : 

¥{\\PBP\\ ^t) ^ ^ yt ^ c.rank{P) andMP orthogonal projection ofW. 

The main result of this paper is the following: 
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Theorem 1.3. Let B be an n x n positive semi-definite random matrix verifying Ei? = 
In and [MSB) for some rj > 0. Then for every e < 1, taking N = Ci(?7)|— ^ we have 
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^ e where Bi, ..,Bn are independent copies of B. 



If X is an isotropic random vector of M", put B = XX* then ||P-BP|| = ||PX|||. 
Therefore if X verifies the property (SR) appearing in [21], then B verifies property 
(M SR). So applying Theorem 1 1.3 1 to B = XX*, we recover the covariance estimation 
as stated in ll2T]| . 

In order to apply our result, beside some examples, we investigate the notion of log- 
concave matrices in relation with the definition of log-concave vectors. Moreover re- 
marking some strong concentration inequalities satisfied by these matrices we are able, 
using the ideas developed in the proof of the main theorem, to have some results with 
high probability rather than only in expectation as is the case in the main result. This 
will be discussed in the last section of the paper. 



The paper is organized as follows: in section 2, we show how to prove Theorem 1 1.3 1 
using two other results (Theorem 12.11 Theorem \2.2) which we prove respectively in 



sections 3 and 4 using again two other results (Theorem 13. 11 Theorem 14. 11 ) whose proofs 
are given respectively in sections 5 and 6. In section 7, we give some applications and 
discuss the notion of log-concave matrices and prove some related results. 

2. Proof of Theorem 11.31 

We first introduce a regularity assumption on the moments which we denote by 

(MWR): 

3p > 1 such that E {Bx, xf ^ Cp Wx e 5""^ 

Note that by a simple integration of tails, (MSR) (with P a rank one projection) implies 
(MWR) with p<l + r]. 
The proof of Theorem 11.31 is based on two theorems dealing with the smallest and 

1 ^ 

largest eigenvalues of -— y^^Bj. 

^ i=i 

Theorem 2.1. Let Bi n x n independent positive semi-definite random matrices verify- 
ing EB, = In and (MWR) . 
Let e < 1, then for 

N ^ 16(16Cp)~ 



2p-l 

e p-i 



we get 

N 

EAm.r 



Ci{r]) = (64c)^+-(l + i)- V64(4c)-(32 + ^)^+- V 256(2c)5 + - (16 + 
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Theorem 2.2. Let B^n x n independent positive semi-definite random matrices verify- 
ing ¥.Bi = In and (MSR). 
Then for any N we have 



Moreover, for e < 1 and N ^ C*2('7)I^tt /?<3ve 
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We will give the proof of these two theorems in sections 3 and 4 respectively. We 
need also a simple lemma: 

Lemma 2.3. Let 1 < r ^ 2 and Zi, Zj^ be independent positive random variables 

1 

with KZi = 1 and satisfying (EZj*") ^ ^ M Then 



E 



N 



N 



i=l 



2M 



Proof. Let {ei)i^N independent ±1 Bemouilli variables. By symmetrization and Jensen's 
inequality we can write 



E 



N 



N 



E^^-1 



i=l 



TV 

E ^i'^i 
i=l 



N 



□ 



Proof of Theorem [73] Take N ^ c{v)~^ satisfying conditions of Theorem 12 . 1 1 (with 
p = 1 + f ) and Theorem IZ21 Note that by the triangle inequality 



N 



N 



i=l 



1 ^ 1 / 1 ^ \ 



N 



-Tr TfE^ 



■=a + 13 



C2{v) = 16c^(32+f )i+t Vl6V2(4c)^+t(8 + f)t 



5 



Observe that 



a = max 



max 



A f- y 5, - -Tr f- y 5i Vn 



N 



-Tr 



i=l 



i=l 



n 



N 



N \ / ^ N 

E Bij - Amin ( ^ E 5i 



Since the two terms in the max are non-negative, then one can bound the max by the 



sum of the two terms. More precisely, we get a ^ Amax I — E -Bi j — A 
and by Theorem |2T| and Theorem 12 .21 we deduce that Ea ^ 2£. 



1 



N 



Note that 



f3 



N ' 



N 

E 



1 ^Tr(5i 



i=l 



n 



N 



N 



i=l 



where Zi = -^^^f^- Since Bi satisfies (MWR), then taking r = min(2, 1 + |) we have 

1 " 

^ iV, (EZ^y ^ -E(E(5.e,,e,r)" ^ c(r/). 



i=i 

Therefore Zi satisfy the conditions of Lemma l2.3l and we deduce that E/3 ^ e by the 
choice of A^. 
As a conclusion 



E 



1 



N 



-Yb, 



^ Ea + E/3 ^ 3£ 



□ 



3. Proof of Theorem 12JJ 

Given A an n x n positive semi-definite matrix such that all eigenvalues of A are 
greater than a lower barrier I a = I i.Q A >- I. In, define the corresponding potential 
function to be = Tt{A - l.In)"^- 

The proof of Theorem 12.11 is based on the following result which will be proved in 
section 5: 

Theorem 3.1. Let A >- I. In cind (t)i{A) ^ (j), B a positive semi-definite random matrix 
satisfying Ei? = /„ and Property {MWR) with some p > \. 
Let £ < 1, if 



1 ^ 



4(8Cp) — 

then there exist I' a random variable such that 
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A + B y I'. In, (pi'iA + B) ^ (t)i{A) and El' ^ I + 1 - e. 



Proof of Theorem IZTI We start with = and /q = — ^ so that ^^^(Ao) 
Applying Theorem l3.11 one can find /i such that 

and 

E/i ^ /o + 1 - £ 
Now apply Theorem |3T| once again to find I2 such that 

and 

E/2 ^ /i + 1 - £ ^ /o + 2(1 - e) 
After steps, we getEAmm(^7v) ^ E^jv ^ ^0 + - Therefore, 



Taking iV = ^, we get EA^i„ ^ 5,, j ^l-2e. 



n 

1^4> 



□ 



4. Proof of Theorem [23] 



Given A an n x n positive semi-definite matrix such that all eigenvalues of A are less 
than an upper barrier ua = ui.Q A -< u.In, define the corresponding potential function 

to be ipu{A) = Tr {u.In - A)~\ 

The proof of Theorem 12.21 is based on the following result which will be proved in 
section 6: 

Theorem 4.1. Let A -< u.In (ind V'u(^) ^ B a positive semi-definite random matrix 
satisfying Ei? = /„ and Property {MSB). 
Let e < 1, if 

there exists u' a random variable such that 

A + B u.In, ^u'{A + B) ^ ipu{A) and Eu' ^ u + 1 + e. 



CM 



8(2c)-(16+f 



16\1- 



16(2c)*+-(8+f)- 
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Proof of Theorem 1272] We start with Aq = Q,uq = ^ so that ipuoi^Q) = i'- 
Applying Theorem|4Tl one can find ui such that 

Ai = Aq + Bi^ ui.In, i^uiiA) ^ ^«o(^o) and Eui ^ uq + 1 + e. 
Now apply Theorem |4. 1 1 once again to find U2 such that 

^2 = ^1 + ^2^ U2.In, ^^^2(^2) ^ i^uA^i) and Ens ^ ni + 1 + 

After steps we get EA^ax ^Yl ^Eun ^ uq + N{1 + e). 
Taking N ^ = c'{rf)~^-^, we deduce that 



e 1 

N 



1=1 



□ 



5. Proof of Theorem 13JJ 



5.1. Notations. We are looking for a random variable /' of the form / + 5 where 5 is a 
positive random variable playing the role of the shift. 

If in addition A :^ (/ + we will note : 
Ls = A-{l + 6).In ^ so that Tr (b^A - (/ + 6).Iny^B^) = b). 



Ai, .., A„ will denote the eigenvalues of A and wi, .., f„ the corresponding eigenvec- 
tors. {vi)i<^n are also the eigenvectors of Lj^ corresponding to the eigenvalues x-^li+s) ■ 

5.2. Finding the shift. To find sufficient conditions for such S exists, we need a matrix 
extension of Lemma 3.4 in fTI which, up to a minor change, is essentially contained 
in Lemma 20 in [9 J and we formulate it here in Lemma 15.21 This method uses the 
Sherman-Morrison- Woodbury formula: 

Lemma 5.1. Let E be an n x n invertible matrix, C a k x k invertible matrix, U an 
n X k matrix and V a k x n matrix. Then we have: 

{E + UCV)-^ = E-^ - E~^U{C'^ + VE-'^uyWE'^ 

Lemma 5.2. Let A as above satisfying A >- I. In. Suppose that one can find S > 
verifying 6 ^ ii. and 



Lf, B 



B^Lj'B^ 



^ 1 



<t>i^siA) - MA) 
Then 

\mUA + B)>l + 5 and (t)i+5{A + B) ^ (l)i{A) . 
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Proof. First note that -p^Tiy = — U so the first condition on 5 implies that 

Amin(^) -^1 + 5. " 

Now using Sherman-Morrison-Woodbury formula with E = Ls, U = V = ^ C = 
In we get : 



s{A + B)=Tt{Ls + B)-' 

= <j)i+s{A) - Tr (lj'B^ (/„ + B^Lj'B^y' B^^Lf 



Lr,B 



1 + 



B^LV^B^ 



Rearranging the hypothesis, we get (f)i+5{A + B) ^ 0z(^)- 



Since \\Lq^\\ ^ Tr (Lq^ 



and 



satisfy conditions of Lemma l5.2[ we may search for 6 satisfying: 



□ 



B^Lx^B^ ^ /Lr \ i?\ then in order to 



(1) 



5^ 



MA) 

For t ^ ^, let us note : 



and 



<f)UA)-MA) 



LJ\B) ^ 1 



and 



q^{t, B) = (L;\ b) = Tr (b{A - (/ + t).Q-' 



q2{t,B) 



Tr(Lr^; 



TTiB{A-il + t).Q-') 
Tr(A-(/ + t)./„)"' 



We have already seen in Lemma|5]2]that if t ^ ^ ^ ijl^ ^^^^ A y {I + t).!^ so the 
definitions above make sense. Since we have : 



(t)i+8{A) - UA) = Tr(A - (/ + 5).Inr^ - Tr(A - 

= 5Tr((A-(/ + 5)./„)-i(A-/./„)-i) 

In order to have ([T]), it will be sufficient to choose 5 satisfying 5 ^ ^ and 

(2) -M5,B)-q^{5,B) ^ 1 



qi and q2 can be expressed as follows: 



^ {Bvi,Vi} ^ l^nx,-i-ty 



qiit.B) = Y. : T , q,{t,B) 



n 

(f)i{A) = ^(Aj — /)~^ ^ so that (Aj — Vj.cj)'^ 1 for all i, then we have : 

i=l 

(1 - t0)(Ai -l) = \i-l- t.{\i - 1)4 ^\i-l-t^\i-l 

therefore 

^ (l-t.0)-igi(O,5) 

and 

(1 - t.0)'g2(O, 5) ^ g2(t, 5) ^ (1 - t0)~'g2(O, 5) 

Lemma 5.3. Lets G (0,1) and take 6 = (1 — s)'^g2(0, i?)l{qi(o,_B)^s}l{g2(o,B)=£|}- ^^^en 
A + B :^ (Z + an^i? </)i+5(A + fi) ^ 

Proof. As stated before, it is sufficient to check that S ^ ^ and |g2 (<5, -B) — Qi {S, B) ^ 1. 
If qi{0,B) ^ s or q2{0,B) ^ | then 5 = and there is nothing to prove since 

MA + B)^(t>iiA). 

In the other case i.e gi(0, B) ^ s and ^2(0, B) ^ |, we have 6 = {1 — 5)^^2(0, B). 
So 5 ^ (1 - ^ ^ and 

□ 

5.3. Estimating the random shift. Now that we have found S, we will estimate E6 
using the property (MWR). We will start by stating some basic facts about qi and ^2- 

Proposition 5.4. Let as above A y I. In and (j)i{A) ^ (J), B satisfying (MWR). Then 
we have the following : 

(1) Egi(0,5) = (piiA) ^^andEqi{0,B)P ^ Cp^P. 

(2) Eq2{0,B) = landEq2{0,B)P ^ Cp. 

(3) P(gi(0,5) ^u)^ Cp{^)PandF{q2iO,B) > u) ^ ^. 
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Proof. Since = /„ then Egi(0, B) = and Eg2(0, B) = 1. 

Now using the triangle inequality and Property (MWR) we get : 



(Egi(o,i?r: 



EE 



vt=l 



{Bvi,Vi) ' 



i=l ^ ^ j=l Aj - / 



With the same argument we prove that Eg2(0, By ^ Cp. The third part of the propo- 



sition follows by Markov's inequality. 



□ 



Lemma 5.5. If S is as in Lemma o3\ Then 

E(5 ^ (1 - sf 



2Cr, 



1 I 1 



Proof. Using the above proposition and Holder's inequality with ^ . 



1 we get 



E(5 = E(l - s)^g2(0,5)l{g^(o,B)<;s}l{g2(0,B)^f} 



Eg2(0, B) - Eg2(0, fi)l{^i(o,B)>s or q2{0,B)>f } 



1 - (Eg2(0, Bfy . P <^ gi(0, 5) > s or ^2(0, B) > - 



C„ 











1 


-c/ 1 


Cp 


(!) 
















1 


— 2Cp 


(!) 













= (1 - s^) 



□ 

Now it remains to make good choice of s and in order to finish the prove Theo- 
rem |3T1 Take I' = I + S, the choice of S being as before with s = |. 
As we have seen, we get A + B >- I'. In and ^^'(A + B) ^ Moreover, 



El' = 1 + K6 > I 



i-2a 



^ 1 



by the choice of (p. This ends the proof of Theorem 13.11 

6. Proof of Theorem 14.11 

6.1. Notations. We are looking for a random variable u' of the form u + A where A is 
a positive random variable playing the role of the shift. 

We will note f/t = (n+t)./„-AsothatTr + t).4 - A)-^^^) = (ur\ B 
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As before, Ai, .., A„ will denote the eigenvalues of A and Vi, ..,Vn the corresponding 
eigenvectors. {vi)i^n are also the eigenvectors of ^7^"^ corresponding to the eigenvalues 



U+t—\i 



6.2. Finding the shift. To find sufficient conditions for such A exists, we need a matrix 
extension of Lemma 3.3 in Q which, up to a minor change, is essentially contained in 
Lemma 19 in [|9l. For the sake of completeness, we include the proof. 

Lemma 6.1. Let A as above satisfying A -< u.In- Suppose that one can find A > 
verifying 



(3) 
Then 



tjjuiA) - iPu+AiA) 



^ 1 



A + B^{u + ^).In and i)u+A{A + B) ^ i)u{A). 
Proof. Since (f/^^, B\ and iJu^A) — ^u+a{.A) are positive, then by ^ we have 



B^Ul^B^ 



< 1 and 



U^\B 



B^2U^^Bi 



First note that 



Ul'BU^^ 



< 1, so U. 'BU.^ -< In. Therefore 



we get B ^ Ua which means that A + B -< (m + A)./„. 

Now using the Sherman-Morrison- Woodbury (see LemmajSTT) with E = U^, U = V 

B^2^C = In we get : 



^u+a{A + B) = Tt{Ua-B)-' 

= ^u+AiA) + Tr (u^'B-^ (in - B-^U^'B^A'' B-^U^' 



^ i^u+A{A) + 



Ua,B 



Ua'BU^^ 



^ MA) 



□ 



and 



We may now find A satisfying Let us note : 

Qi{t,B) = \\B^2Uf^B^\ = \\B^{u + t).In- Ay^ B^ 

(Ur',B) Tr (Bi{u + t).In- AY 

Q2{t, B) 



MA) - Mt{A) MA) - Mt{A) 

Since Qi and Q2 are both decreasing in t, we work with each separately. Precisely 
fix 6* e (0, 1) and define Ai, A2 as follows : 
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Ai the smallest positive number such that Qi{Ai, B) ^ 9 



and 



A2 the smallest positive number such that (52(^2, B) ^ 1 — 9 

Now take A = Ai + A2, then Qi(A, 5) +Q2(A, 5) ^9+1-9=1. So this choice 
of A satisfies and it remains now to estimate Ai and A2 separately. 



6.3. Estimating Ai. 



We may write Qi(Ai, B) 



^ u + Ai - Ai 

Put = B^Viv\B^, Hi = ip{u — \i) and fi = tpAi. Denote Ps the orthogonal pro- 
jection on {vi)i(zs, clearly rank{Ps) = \S\. Then we have : 



r E|ie.i 



p 



^t]=T{\\PsBPs\\^t)^j^ ^t^c\S\ 



y 1 ^u{A) ^ ^ 

/i is the smallest positive number such that — :< —Id 

We will need an analog of Lemma 3.5 appearing in [|2T]| . We extend this lemma to a 
matrix setting: 

Lemma 6.2. Suppose {^j}j^n <^fe symmetric positive semi-definite random matrices 
with ]E||^j|| = 1 and : 



P 



ie5 



^M^Ti^ provided t>c|^| = c^E||ei||. 



for all subsets S C [n] and some constants c,ri > 0. Consider positive numbers pi such 
that 

n 1 
i=l A^* 

Let fi be the minimal positive number such that 



f^l^^^ + l^ 

for some K ^ C = Ac. Then E/i ^ 



^ K ■ Id, 



■ 
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We do not reproduce the proof here as it is a direct adaptation of the argument in [[211 
to the matrix setting. Applying Lemma l6^ we get E/i < 0(77) (jfj '^^ , so that 

(4) EAi^c(r/)- 



Ql+V 

6.4. Estimating A2. 



Suppose 9^1- Since ipuiA) - ^pu+t{A) = t.Tr - A) ^ ((u + t).4 - A) ^ 



we can wnte 



(5) (52(i -B) = i^+t-^i)^ < 1 («+i-AO(«-AO 



■.= jP2it,B) 

First note that P2(i, B) can be written as Y.i Oii{t) {Bvi, Vi) with X]j = 1- Having this 
in mind, one can easily check that EP2(^! B) = 1 and 



32 
4 • 



(6) EP2(t,5)'+^ ^c(r/), 

where for the last inequality, we used the fact that B satisfies {MWR) with p = 1 + 
In order to estimate A2, we will divide it into two parts as follows: 

A2 = A2l{p^(o,B)^^| + A2l{p^(o,B)>^} := Hi + H2 

Let us start by estimating Eifi. Suppose that P2(0, B) ^ and denote 

X = (1 + 4e)P2(0,P). 

Since ijjuiA) ^ ■?/;, wehave (m— Aj).-?/^ ^ 1 Vi and therefore w+x—Aj ^ {l+xip){u—Xi). 
This implies that 

P2(x,P) ^ (l + a;V^)P2(0,P). 

Now write 

Q^ix, B) ^ ip2(x, B) ^ i^P2(0, B) ^ ^"^f^+J^^^ ^ 1 - ^, 
X X 1 + 4t^ 

which means that 

A2l|p,(o,BK^}^(l + 4^m(0,P) 

and therefore 

(7) Ei/i = EA2l|p^(o^5)^^| < 1 + 40 

Now it remains to estimate Ei72- For that we need to prove a moment estimate for 
A2. First observe that using we have 

p{A2 >t} = p{g2(t, B)>i-e}^ p{P2(t, B)>t.{i-9)}^ ^ 
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By integration, this implies 

Jo 2 Jo 2 J I 

Let p' = 1 + f , applying Holder's inequality with + ^ = 1 we have : 



(8) = EA2l|p^(o^5)>^| ^ (eA^') ^' [P |P2(0, B) > ^ 



47/; 



^ c(r/) 

Looking at dV]) and ([8]) we have 



V 



EA2 ^ 1 + 45 + c(r/) (^^ j ' 
Putting the estimates of Ai and A2 together we deduce 

EA^l + 45 + c(7;) (^^y +c(r/)^ 

We are now ready to finish the proof. Take u' = n + A, A being chosen as before with 

0=1- Then taking ip = c{r])e with the constant depending on rj properly chosen, 
we get EA ^ 1 + e. 



7. Applications 

In this section, we will show how to apply our main result. After giving examples, 
we will discuss in details the case of log-concave matrices for which we give results 
with high probability estimates. Let us first replace {MSB) with a stronger, but easier 
to manipulate, property which we denote by {MSB*). If 5 is an n x n positive semi- 
definite random matrix such that Ei? = Id, we will say that B satisfies {MSB*) if for 
some ?7 > 0: 

¥{Tt{BB) ^t) ^ yt ^ c.rank{B) and VP orthogonal projection of M". 

Note that since ||PPP|| ^ Tr {BBB) = Tr {BB), then {MSB*) is clearly stronger 
than {MSB). 
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7.1. (2 + £:)-moments for the spectrum. Looking carefully at (MSR), one can see 
that it implies regularity assumptions on the eigenvalues of B. Putting some indepen- 
dence in the spectral decomposition of i?, we will only need to use the regularity of the 
eigenvalues. To be more precise, we have the following: 

Proposition 7.1. Let B = UDU* be an n x n symmetric positive semi-definite ran- 
dom matrix. Denote {<yj)j^n the diagonal entries of D. Suppose that U and D are 
independent and that {<yj)j<:n are independent and satisfy the following: 

\/i ^ n, Eoj = 1 and (Eaf)p ^ c, 
for some p > 2. Then B satisfies (MSR*). 

Proof First note that since U and D are independent and Eoj = 1 then KB = Id. Let 
k > and P be an orthogonal projection of rank k on M", then Q = U*PU is a random 
orthogonal projection of rank k independent of D. Note that Tr {PB) = J^i^nQu'^u 
and now using Markov's inequality we have for t > k, 

p 



P {Tr (PB) ^t} ^ 



1 



-E 



a,: 



{t - k)P 

Looking at the expectation with respect to D and using Rosenthal's inequality (see f\M ) 
we get 



E 



D 



2 



^ C{p) max - 1 r, E ^'^^1 



a,; 



Taking in account that qu ^ 1, which implies that for any I ^ l,Y.i lii ^ k, we deduce 
that 



E 



^ C{p)k^ 



Instead of Rosenthal's inequality, we could have used a symmetrization argument along- 
side Khintchine's inequality to get the estimate above. 

One can easily conclude that B satisfies {MSR*) with r/ = | — L □ 



Applying Theorem 1 1.31 we can deduce the following proposition: 

Proposition 7.2. Let B = UDU* be an n x n symmetric positive semi-definite ran- 
dom matrix. Denote {otfjj^n the diagonal entries of D. Suppose that U and D are 
independent and that {otj)j^n <^^^ independent and satisfy the following: 

yi ^ n, Ettj = 1 and (Eaf)p ^ c, 
for some p > 2. Let e < 1, then taking N = C{p) we have 

N 



E 



i=l 



^ e where Bi, ..,Bn are independent copies of B. 
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7.2. From (SR) to (MSR). A random vector X in M} is called isotropic if its covari- 
ance matrix is the identity i.e EXX* = Id. In |[2T]| . an isotropic random vector X in 
was said to satisfy (SR) if for some rj > 0, 

P (IIPXII2 ^ t) ^ ^ Vt ^ c.rank{P) and VP orthogonal projection of 

We will show how to jump from this property dealing with vectors to the property 
(Af SR*) dealing with matrices. 

Proposition 7.3. Let A be an n x m random matrix and denote by {Ci)i^rn ifs columns. 
Suppose that A'^ = ^Jra{C\ , . . , ) is an isotropic random vector in M"'" which satisfies 
property {SR). Then B = AA* verifies = J„ and Property {MSR*). 

Proof. For / ^ nm, one can write I = {j — l)n + i with 1 ^ i ^ n, 1 ^ j ^ m. So 
that the coordinates of A' are given by a'l = ^/mai^j, and since A' is isotropic we get 

m 

The terms of B are given by hi^ = ^ ai^sCt'j,s- We deduce that 'Ebij = 6ij and therefore 

s=l 

EB = /„. 

Let P be an orthogonal projection of M" and put P' = 1.^ ® P. Clearly we have 

WP'A'Wl = mTT{PB) Sindrank{P') = m.rank{P). 

Let t ^ c.rank{P) then mt ^ c.rank{P') and by property (S*/?) we have: 

This means that 

and therefore P satisfies {MSR*). □ 

Remark 7.4. In ll2T]| . wa* shown that an isotropic log-concave vector satisfies {SR). 
We will discuss with details this notion for matrices in the next section. 
In [[II, it was shown that, with r properly chosen, an isotropic {—^) -concave random 
vector satisfies {SR). Therefore, one can adapt the results of the next section to the case 
of {—^) -concave random matrices. 

7.3. From log-concave vectors to matrices. In this section, we will discuss in details 
the case of log-concave matrices. After giving some tails estimate, we deduce results 
with high probability. The methods will use tools developed in the proof of the main 
theorem. Finally, we give an example of log-concave matrices and apply the results 
obtained to these matrices. 

Definition 7.5. Let A be an n x m random matrix and denote by {Ci)i^rn its columns. 
We will say that A is an isotropic log-concave matrix if A'^ = ^/rn{Cl, .., C^) is an 
isotropic random vector in M""^ with a log-concave density with respect to the Lebesgue 
measure in M""^. 
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Remark 7.6. Let {aij) the entries of A. Saying that A' is isotropic means that 

m 

This implies that for any n x m matrix M we have : 

E (A, M)A = ETr (A^m) A = —M. 

^ ^ m 

One can view this as an analogue to an isotropic condition in the vector case: in fact if 
A = X is a vector (i.e ann x 1 matrix), the above condition would be 

E{X,y)X = y forallyeW. 

which means that X is isotropic in M". 

Proposition 7.7. Let Abe ann xm isotropic log-concave matrix and denote B = AA^. 
Then for every orthogonal projection P on M" we have a large deviation estimate for 
Tt{PB) 

P {Tr{PB) ^ cit} ^ exp (- Vtm) Vt ^ rank{P). 
and a small ball probability estimate 

P {Tr(P5) ^ C2e.rank{P)} <: £^2^/m.rank{P) ^ 1. 

Proof. Let P an orthogonal projection on M" and denote P' = Im® P- As we have seen 
before Tr(Pi?) = ||PA||^g = ^||P'^'||2 and ran/c(P') = m.rank{P). Using Paouris 
result [|T6]| for the isotropic log-concave vector A', we have 

P {||P'A'||2 ^ ciu} ^ exp (-v^) Vn ^ rank{P'). 

Let t ^ rank{P) and write u = t.m. Since u ^ m.rank{P) = rank{P') we have 

P{m.Tr(PP) ^ Cit.m} ^ exp (^—^/t.nij 

which gives the large deviation estimate stated above. 

For the small ball probability estimate, we use once again a result of Paouris [[TTII 
dealing with isotropic log-concave vector: 

P{||P'A'||^ ^ C2e.rank{P')} <: e^^V^^^^) \/e <: 1. 

Writing this in terms of B and P, we easily get the conclusion. □ 

Now we will apply Theorem 1 1.3 1 to this class of matrices. We get the following: 

Proposition 7.8. Let A be an n x m isotropic log-concave matrix. Then B = AA^ 
satisfies {MSB). Moreover We > 0, taking N > c{e)n independent copies of B we 
have 

_ 1 _ 
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Proof. Note first that since A is isotropic in the sense of definition 17 .51 then B = AA* 
satisfies EB = In- 

By proposition 17 .7 [ B satisfies 

P (Tr(PB) ^ cit) ^ exp{-Vtm) Vt ^ rank{P) and VP orthogonal projection of M". 

and therefore (MSR). Applying theorem [T3] we deduce the result. □ 

The probability estimate for these log-concave matrices are strong enough which al- 
lows us to obtain some results with high probability rather than in expectation as is the 
case above. Precisely, we can prove the following : 

Proposition 7.9. Let p > and N ^ n'^, A an isotropic log-concave matrix and 
B = AA^. Ifm ^ [2(1 + p) \og2nf, then with probability ^ 1 — exp{—}^^/m) we have 



Proof. The proof of proposition l7.9l follows the same idea as in the previous section. 
We will only need the following property satisfied by our matrix B = AA* (using 
Proposition 17 .7 1 for a rank 1 projection): 

P ((Px, x) ^ cit) ^ exp{-Vtm) Vt ^ 1 and Vx G §""\ 

Recall some notations : 

N 

Ao = 0, = Pi, ^2 = + Pi, .., An = An^i + Bn = J2 Bi. uo = 2ci.n, u, = 

i=l 

uq + 2ci, U2 = ui + 2ci, .., UN = U]s[-i + 2ci and Ui = Ui+iln — Ai, A = 2ci plays 
the role of the shift which will no longer be random. Note ipuX^i) = Tr {ui — A/)^^ 
the corresponding potential function when Ai -< Ui.In and if) = ijjuo{Ao) = 
Denote by the event "A^ -< and ^uii^i) ^ 

Clearly P (Q'o) = 1- Suppose now that is satisfied, as we have seen in Lemma[6T| 
the following condition is sufficient for the occurrence of the event $5.j+i : 

g2(2ci,P,+i) +Qi(2ci,Pi+i) ^ 1 

Note that Q2(2ci, Pj+i) ^ 2^P2(2ci, Pj+i), where P2 is defined in Q. Now denot- 
ing \j the eigenvalues of A^ and Vj the corresponding eigenvectors, taking the probabil- 
ity with respect to Pj+i one can write : 
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P (g2(2ci, + gi(2ci, > 1) ^ P (^^2(2ci, + gi(2ci, > 1^ 

^ P(3j7 > Ci) + P Mj/ {B,+^v„v,) > ^ 



^ 2n.exp{—y/m) 

So we have shown that P ^ 1 — 2n.exp{—y/m). Since are independent 

we have : 



P (^A^ax (^E ^ 2ci.(n + iV) j ^ P (53^) 



^ 1 — 2Nn.exp{—\/m) 



Propo sition l7 .9 1 follow s by the choice of m. Note that the same method works when- 
ever we have in (MSR) a probability less than 

□ 

Now using the small ball probability estimate alongside the large deviation given by 
Proposition 17 ■?[ we have also an estimate on the smallest eigenvalue. 

Proposition 7.10. Let p > and < N ^ n^'^P, A an n x m isotropic log- 

r n 2 

concave matrix and B = AA^. If m ^ ^(2 + p) log(2n) , then with probability 
^ exp(— y-y/m) we have 

Kin (T.^^ ^-Cm+jN 

Proof. Here we only need the following two properties satisfyied by our matrix B : 

¥{{Bx,x) ^ at) ^ exp{-Vtm) Vt ^ 1 and Va; G §"~^ 
and taking e = | in Proposition 17 .71 

P (^{Bx,x) ^ ^ exp{-C2Vrn) Wx G 
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Recall some notations : 

TV 

Aq = 0, Ai = 5i, = Ai + 5i, An = Am^i + Bn = Y. Bi- k = -ci.n, h = 

1=1 

^o + f ' ^2 = h + f, ■■, In = h-i + f andLj = Ai-k+J^, 6 = f plays the role of the 
shift which will no longer be random. Note (piX^i) = Tr (Aj — the corresponding 
potential function when Ai >z k-In and = (pi^^Ao) = ^. Note also that 5^0 since 
ci > 1 while C2 < 1 for obvious reasons. 
Denote by the event "A^ >z U-In and (pX^i) ^ "• 

Clearly P (^^o) = 1- Suppose now that Q^j is satisfied, following what was done after 
Lemma 15^ condition Q was sufficient for the occurrence of the event 53^+1 : 

— g2(x' B^+l) - Bi+i) ^ 1 

C2 4 4 
Denoting \j the eigenvalues of A^ and Vj the corresponding eigenvectors, taking the 
probability with respect to one can write : 

P(lg2(|,5m)-gi(|,i?m)<l) ^ 



2 . 
^ 2n.exp{—C2\/rn) 

So we have shown that P (53,/+ 1 1 53 j) ^ 1— 2n.exp(— C2ym)- Since i?j are independent 
we have : 



^ P {^nI^n^i) P (5>tv-i|5>tv-2) ..P (^o) 
^ 1 — 2Nn.exp{—C2\/rn) 
Proposition !? . 1 01 follows by the choice of m. 

□ 

Combining the two previous propositions we get the following: 

Corollary 7.11. Let A be an n x m isotropic log-concave matrix. There exists c,C>0 
universal constants such that if m ^ [Clog(2n)]^, taking N = Cn copies of A then 
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with probability ^ exp(— Ci/m) we have 




7.3.1. Example of log-concave matrices. 

For X e M'^, we denote by x the vector with components arranged in nonincreas- 
ing order. Let / : M'^ — )■ M, we say that / is absolutely symmetric if f{x) = f{x) for 
all X E M^. (For example, || ■ \\p is absolutely symmetric). 

Define F a function on M„^„ by F{A) = / (si(A), .., Sfc(A)) for A G M„,^ and 
k = min(?7,,m). It was shown by Lewis lfT3l that / is absolutely symmetric if and 
only if F is unitary invariant and of this form. Moreover, / is convex if and only if F is 
convex. 

Let y4 be an n X m random matrix whose density with respect to Lebesgue measure 
is given by G{A) = exp (— / {si{A), .., Sk{A))), where / is an absolutely symmetric 
convex function. By the remark above, G is log-concave. This covers the case of ran- 
dom matrices with density of the form exp (— J2i ^ {si{A))) where V is an increasing 
convex function on IR+. When V{x) = x^, this would be the gaussian unitary ensemble 
GUE. 

Let (aj,j) the entries of A. By a good normalization of / we can suppose that A satisfies 

To see this, fix (i, j) and (/c, /) two different indexes. Note Dj = diag{l, .., —1, .., 1) the 
m X m diagonal matrix where the —1 is on the j*'* term. Let -E(j,fc) is the n x n matrix 
obtained by swapping the i*^ and A;*^ rows in the identity matrix. Note also F(j the 
m X m matrix obtained by swapping the j*'' and /*^' rows in the identity matrix. 

It is easy to see that ADj change the j*^ column of A to its opposite and keep the rest 
unchanged. Note that ADj has the same singular values as A. 

Similarly, E(^i^k)AF(^j^i) permute Oj j with ^ and keep the other terms unchanged. 
Note also that E(i^k)^F(^j,i) has the same singular values as A. 

Finally note that these two transformations has a Jacobian equal to 1, and since / 
is absolutely symmetric these transformations which preserve the singular values don't 
affect the density. 

If j 7^ /, by a change of variables M = ADj the density is invariant and we have 

j aijak,iG{A)dA = - J aijak,iG{A)dA 
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Doing the change of variables M = DiA when i ^ k,we can conclude that 

Eaijak,i = if{ij)^{k,l) 
Now by a change of variables M = E(^ij)AF(^k^i) the density is invariant and we have 

/ alG{A)dA = J alG{A)dA 

This implies that 

/ alG{A)dA = ^ I <iG{A)dA = — [ \\A\\l,G{A)dA 

Now we may normalize / in order to make the previous term equal Suppose that 

1 



n 



\Aru,G{A)dA 



Define f{x) = f{^cx) - nm\og{^c) and G{A) = exp ..,su{A))). 

Note that G is a probability density. Indeed, by the change of variables M = y/cA 
we have 



G{A)dA = I exp [-f{V~csi{A), .., V^Sk{A))) {^cT'^dA 
exp{-f{si{M),..,Sk{M)))dM = 1 



Note also that G satisfies our isotropic condition. Indeed, by the same change of vari- 
ables we can write 

^ / \\A\\l,GiA)dA = i- / \\M\\l,G{M)dM = 1 

As a conclusion, we can deduce that such matrices are isotropic log-concave. More- 
over, since in this case EA*A = ^Im then we also have that ^J^A'^ is an m x n isotropic 
log-concave matrix. We summarize this in the following proposition: 

Proposition 7.12. Let A be an n x m random matrix whose density with respect to 
Lebesgue is given by 

G(A) = exp(-/(si(A),...,Sfc(A))), 

where f is an absolutely symmetric convex function, properly normalized as above and 
k = min(n, m). Then A is an isotropic log-concave matrix, and \J^A^ is an m x n 
isotropic log-concave matrix. 

Remark 7.13. In a similar way, one can prove that taking A an n x m random matrix 
whose density with respect to Lebesgue is given by 

G{A) = {f{sM),-,Sk{A))p'^'^\ 
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where f is an absolutely symmetric convex function, properly normalized, r chosen as 
in and k = min(r2, m), then A is an isotropic {—^)-concave matrix. 

Applying Proposition 17 . 1 01 and Propo sition 17 . 9 1 for A and A* we get: 

Proposition 7.14. Let A be an n x m random matrix whose density with respect to 
Lebesgue is given by 

G(A)=exp(-/(si(A),...,s,.(A))), 

where f is an absolutely symmetric function, properly normalized as above and k = 
min(?7,, m). 

Suppose that n ^ [Clog(2m)]^ and m ^ [C log(2?T,)]^, taking N = Cmax(n,m) then 
with probability ^ 1 — exp(— ca/A;) we have 



and 
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